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Pr.gface 


The  objective  of  this  research  was  to  perform  an 
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thank  my  parents  for  their  support  and  prayers  as  I 
completed  the  project. 

Most  of  all,  I  want  to  thank  my  wife  Dawn,  for  her 
understanding  and  love  as  I  struggled  through  many  a  late 
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Abstract 

The  research  was  performed  to  explore  the  extent  that 
electronic  delivery  of  data  is  being  used  within  the 
Aeronautical  Systems  Division  (ASD)  and  the  Human  Systems 
Division  (HSD) .  Surveys  were  administered  to  acquisition 
personnel  at  ASD  and  HSD  to  determine  the  data  management 
and  computer  training  that  they  have  received,  the  computer 
resources  available  to  them  and  whether  or  not  these 
individuals  are  working  on  programs  that  require  the 
electronic  delivery  of  data.  Additionally,  a  telephone 
interview  was  conducted  with  the  points  of  contact  for  109 
contracts  awarded  within  ASD  from  Oct  89  to  May  91  to 
determine  electronic  data  delivery  usage  on  current 
contracts . 

The  results  indicate  that  59%  of  the  individuals  are 
trained  in  data  management,  and  only  29.5%  were  able  to 
successfully  choose  the  correct  DODI  5000.2  requirement 
concerning  the  digital  delivery  of  data.  Of  the  109 
contracts  reviewed  less  than  10%  were  using  electronic 
delivery  of  contractor  data.  The  most  common  barriers  to 
electronic  delivery  of  data  mentioned  by  respondents 
included  lack  of  guidance,  resources,  and  training  as  well 
as  working  with  small  businesses.  A  positive  aspect  is  that 
11.7%  of  the  survey  respondents  stated  that  future  contracts 
will  require  the  electronic  delivery  of  data. 
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AN  ANALYSIS  OF  FACTORS  AFFECTING 
THE  ELECTRONIC  DELIVERY  OF 
CONTRACTOR  DATA 

I«  Introduction 


General  Issues 

Throughout  the  acquisition  process,  there  is  a  high 
dependence  on  paper  to  coninunicate  ideas  and  document 
program  objectives  and  findings.  Yet,  the  process  to  get  to 
a  final  dociunent,  a  request-f or-proposal ,  or  a  technical 
data  package  is  time  consvuning  and  requires  excessive 
amounts  of  paper.  General  Yates,  Commander  of  Air  Force 
Systems  Command,  emphasized  that  the  acquisition  process  is 
inundated  by  paper  and  that  the  Air  Force  must  take 
advantage  of  current  technology  to  provide  data  when  needed, 
where  needed,  and  in  the  proper  format.  (22:1) 

Problem  Statement 

There  is  strong  emphasis  within  the  Department  of 
Defense  (DOD)  to  fully  exploit  the  use  of  computers 
throughout  the  acquisition  process,  beginning  with 
requirements  definition  and  continuing  through  the  entire 
life  of  the  system.  DOD  Instruction  (DODI)  5000.2,  "Defense 
Acquisition  Program  Procedures,"  requires  that  contract  data 
deliverables  be  prepared  and  used  in  digital  form  unless  it 
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is  not  cost  effective  for  the  government  (9:9B1). 
Additionally,  the  Air  Force  Acquifition  Executive  states  in 
a  letter  dated  Nov  17,  1990,  "In  addition  to  the  specific 
reporting  requirements,  it  would  seem  that  virtually  all 
docximents  which  flow  between  the  program  offices  and 
contractors  could  be  paperless."  (21:1). 

Although  there  are  highly  visible  and  established 
requirements  to  use  electronic  delivery  of  contract 
submittals,  there  are  still  many  obstacles  to  overcome 
before  a  paperless  data  delivery  system  can  be  fully 
implemented,  This  research  will  attempt  to  identify 
these  obstacles  and  where  possible,  identify  solutions. 

Objective 

The  objective  of  this  research  is  to  perform  an 
exploratory  study  to  determine  the  extent  that  electronic 
delivery  of  contractor  data  is  being  used  on  contracts 
within  the  Aeronautical  Systems  Division  (ASD)  and  the  Human 
Systems  Division  (HSD) .  The  delivery  of  Contract  Data 
Requirements  List  (CDRL)  data  as  well  as  non-required 
contractual  data  will  be  investigated.  Specific  objectives 
are  as  follows: 

1.  Analyze  the  extent  to  which  electronic  data 
delivery  is  being  used  by  program  offices  within  ASD  and 
HSD. 
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2.  Identify  the  current  requirements  concerning 
digital  delivery  of  contractor  data  and  the  anticipated 
benefits  of  using  digital  data  versus  paper  copies. 

3.  Identify  problem  areas  or  perceived  problem  areas 
which  prevent  the  effective  use  of  electronic  data  delivery. 

4.  Determine  what  lessons  can  be  gained  from  those 
program  offices  that  have  implemented  or  are  attempting  to 
implement  the  electronic  delivery  of  contractor  data? 

Investigative  Questions 

To  meet  the  research  objectives,  the  following  research 
questions  must  be  answered. 

1.  Are  program  office  personnel  aware  of  the  DOD 
requirements  concerning  digital  data  delivery? 

2.  Are  the  necessary  program  office  personnel  trained 
in  data  management? 

3.  Are  the  necessary  program  office  personnel  trained 
in  the  use  of  Electronic-Mail  (E-Mail)? 

4.  Are  personnel  trained  in  the  use  of  networked 
computer  systems  (Local  Area  Networks  -  LANS)? 

5.  Is  electronic  data  delivery  being  applied  to 
current  Air  Force  acquisition  programs  within  ASD? 

6.  Is  electronic  data  delivery  applied  to  all  data 
items  on  contract?  If  not,  which  data  items  are  received 
eletronically? 

7.  When  digital  delivery  is  not  applied.  Why  is  it  not 
applied? 
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8.  When  digital  delivery  is  applied  to  a  contract, 
what  problem  areas  are  encountered?  What  benefits  are 
realized? 

9.  How  is  electronic  delivery  of  data  accomplished? 

10.  Are  any  additional  forms  of  contract 
correspondence  electronically  transferred  between  the 
government  and  contractor? 

Scope  of  Research 

There  is  a  push  within  the  DOD  to  develop  a  completely 
paperless  acquisition  process,  from  requirements  definition 
to  request  for  proposal  preparation  through  source  selection 
and  then  into  contract  execution.  This  research  will  focus 
on  the  latter  phase  --  contract  execution;  particularly,  the 
delivery  of  contract  data  deliverables. 

Summary. 

The  purpose  of  this  chapter  was  to  provide  a  brief 
overview  of  the  issues  associated  with  the  DOD's  move  toward 
a  paperless  acquisition  process.  Additionally,  it  presented 
the  problem  statement,  objective  of  this  study,  and  specific 
research  questions.  Chapter  II  will  provide  a  more  detailed 
review  of  the  literature  concerning  the  electronic  delivery 
of  data.  Chapter  III  is  a  discussion  of  the  methodology 
used  to  carry  out  this  research.  Chapter  IV  is  a  summary  of 
the  results  and  Chapter  V  contains  conclusions  and 
recommendations  based  on  the  study's  findings. 
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II.  Literature  Review 

This  chapter  provides  background  concerning  the 
electronic  delivery  of  data.  Electronic  delivery  has 
several  connotations  which  will  be  discussed;  among  these 
are  electronic  data  interchange  (EDI)  and  electronic-mail 
(E-mail) . 

Key  Terms 

Data:  Recorded  information,  regardless  of  form  or 
characteristic  (10:A8). 

Technical  Data:  Recorded  information,  regardless  of  form  or 
characteristic,  of  a  scientific  or  technical  nature  (10:A8). 

Contract  Data  Requirements  List  (CDRL) :  A  list  of 
deliverable  data  requirements  that  is  authorized  for  a 
specific  acquisition  and  made  a  part  of  the  contract  (7:8). 

Electronic  Data  Interchange  (EDI):  "A  computer  to  computer 
and  application  to  application  interchange  of  business 
transactions"  (1:222). 

Based  on  the  definition  of  data,  whether  the 
information  delivered  to  the  government  is  on  paper  or  in 
digital  format,  it's  data.  The  key  then  is  to  determine  the 
most  efficient  method  to  get  data  delivered  to  the 
government.  Because  of  advances  in  technology  and  the 
sharing  of  data  bases  it  may  be  more  appropriate  to  get  the 
government  to  the  data.  There  are  at  least  three  efforts 
underway  by  the  DOD  to  begin  the  process  of  electronic 
delivery  of  data.  These  are  Computer-aided  Acquisition 
Logistics  Support  (CALS) ,  Contractor  Integrated  Technical 
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Information  Service  (CITIS)  which  is  a  sxibset  of  CALS,  and 
Electronic  Coxnmerce  through  EDI. 

Each  of  these  will  be  reviewed;  but  emphasis  will  be 
placed  on  Electronic  Commerce  as  this  relates  more  to  the 
everyday  correspondence  that  flows  between  two  business 
partners.  In  the  case  of  government  contracts,  the  two 
business  partners  are  the  US  government  and  the  particular 
contractor. 

In  the  preceding  paragraph,  government  is  used 
primal  ^  indicate  the  system  program  office  or 
government  contracting  activity.  These  entities  have 
regular  contact  with  the  contractor;  whereas  some  functional 
representatives  and  using  organizations  only  receive 
contract  data  when  it  is  time  for  it  to  be  reviewed. 
Therefore,  one  must  keep  in  mind  that  an  efficient  system 
must  allow  for  delivery  to  multiple  addresses. 

The  review  process  of  contractual  data  begins  with  the 
contractor  collecting  information  and  formatting  the 
information  in  accordance  with  the  CDRL.  The  data  is  then: 

1.  Mailed  to  the  government  and  depending  on  the 
method  used  for  mailing,  the  data  is  received  one  to  ten 
days  later. 

2.  Copies  are  made  and  distributed  for  review  and 
comment.  If  sufficient  copies  were  received  from  the 
contractor  this  step  is  not  necessary;  however,  comments 
must  still  be  solicited  by  the  office  of  primary 
responsibility  (OPR)  for  that  CDRL  item. 
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3.  The  comments  are  received  and  consolidated  by  the 

OPR. 

4.  Comments  are  sent  to  the  contractor  for 
incorporation  into  the  next  stibmittal  of  that  CDRL  item. 

5.  The  process  starts  over  again. 

Each  step  in  this  process  can  be  improved  and  expedited 
using  electronic  delivery.  Application  of  EDI  or  other 
similar  information  technologies  could  significantly  improve 
the  timeliness  of  the  contracting  process,  increase 
competition,  and  lower  the  cost  of  contracting  efforts  (6:6- 
2-1) . 

Computer  Aided  Acquisition  Logistics  Support  fCALS^ 

"CALS  is  a  DOD  and  Industry  strategy  to  effect  the 
transition  from  the  current  paper- intensive  design, 
manufacturing  and  support  processes  to  a  highly  automated, 
integrated  mode  of  operation  for  future  weapon  systems 
(10:IV,  8:12).  Originally  CALS  was  a  DOD  strategy  only,  but 
industry  was  later  added  when  it  was  realized  that  industry 
was  half  the  team  and  played  an  important  role. 

On  24  September  1985,  the  Deputy  Secretary  of  Defense 
issued  a  memorandum  that  defined  the  key  objectives  of  CALS. 
These  objectives  have  been  modified  slightly  to  emphasize 
the  program  life-cycle  and  are  as  follows: 

1.  To  accelerate  the  integration  of  reliability  and 
maintainability  design  tools  into  contractor 
computer-aided  design  and  engineering  systems. 
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2.  To  encourage  and  accelerate  the  automation  and 
integration  of  contractor  processes  for  generating 
weapon  system  technical  data  in  digital  form. 

3.  To  rapidly  increase  DOD's  capabilities  to  receive, 
store,  distribute,  and  use  logistic  technical  data  in 
digital  form  to  improve  weapon  system  maintenance,  training, 
and  spare  parts  reprocurement  (10: IV, 8: 12) . 

CALS  is  a  term  that  is  associated  with  all  forms  of 
electronic  delivery  of  data.  However,  the  main  focus  of 
CALS  is  to  facilitate  the  integration  of  digital  technical 
information  only.  Technical  is  underlined  to  emphasize  the 
fact  that  the  definition  of  technical  data  includes  only 
that  data  that  is  of  a  scientific  or  engineering  nature  and 
does  not  include  computer  software  or  administrative  data 
such  as  financial  and  or  management  information  (10:45). 
However,  the  basic  assumptions  and  concepts  of  CALS  are 
similar  to  those  of  CITIS  and  EDI.  MIL-HDBK-59A  describes 
the  CALS  system  of  systems  approach  which  consists  of  four 
key  elements: 

1.  Industrial  systems,  (i.e.,  design,  manufacturing, 
and  customer  support) . 

2.  Government  systems,  (i.e.,  acquisition  and  logistic 
support) . 

3.  Interfaces  between  industzry  and  government. 

4.  Interfaces  within  industry  among  prime  contractors, 
subcontractors,  and  vendors  (10:52). 

Elements  not  accounted  for  in  this  systems  approach 
are  the  interfaces  within  the  government  among  program 
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offices,  users,  and  Air  Logistics  Centers.  These  interfaces 
exist  to  a  limited  degree  through  the  Defense  Data  Network 
(DDN)  and  will  be  discussed  later. 

There  are  three  broad  groups  of  requirements  documents 
that  constitute  the  CALS  interchange  standards  (10:52). 

1.  Functional  Standards  >  tailored  versions  of  existing 
MIL>STDs  and  specifications  and  contractual  statements 
of  work. 

2.  Technical  Standards  -  relevant  conventions  for  the 
management,  formatting,  and  physical  or 
telecommunications  exchange  of  digitized  text, 
graphics,  alphanumerics  and  other  forms  of  data. 

3.  Data  Standards  -  data  dictionaries  and  other 
conventions  needed  for  access  to  data  bases 
(8:15,10:52) . 

Contractor  Integrated  Technical  Information  Services  rciTIS) 
There  are  three  methods  for  obtaining  data  through 
government/ industry  interchange.  These  three  methods  are 
hardcopy  documents,  digitized  processable  files,  and 
interactive  access.  This  interactive  process  is  rapidly 
evolving  into  CITIS. 

CITIS  are  contractor  developed  and  managed  information 
services  that  are  made  available  for  automated  access  and 
transfer  to  authorized  users  with  a  need  to  know  (10:54). 
Through  CITIS,  data  that  is  applicable  to  more  than  weapon 
systems  must  be  available  to  users  without  unnecessary 
storage  redundancy.  CITIS  is  an  interactive  service  with 
users  who  may  be  involved  with  similar  weapons  systems; 
therefore,  standardization  among  data  bases  is  essential  to 
allow  for  the  free  interchange  of  data.  This  means  that 
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each  contract  must  identify  to  the  contractor  the 
requirements  for  access  to  and  interaction  (from  and  to  the 
government)  with  the  data  base  (system  availability,  maximum 
response  time,  equipment,  etc...)  as  well  as  defining  the 
data  product  in  terms  of  the  data  elements  and  data 
relationships  that  comprise  the  data  base  to  be  accessed 
(10:77) . 

Electronic  Data  Interchange  ^EDI) 

EDI  is  the  process  in  which  computers  talk  to  computers 
to  pass  information  automatically,  electronically  with 
little  or  no  human  interaction.  EDI  is  common  in  the 
commercial  sector,  especially  for  the  transfer  of  funds 
(1:225).  For  example,  many  companies  including  the  Federal 
Government  encourage  the  use  of  direct  deposit  of  paychecks. 
Why?  Because  it  costs  less  and  is  more  efficient.  Rather 
than  send  a  check  to  each  individual  employee,  the  employer 
(company)  collects  pay  deposits  by  financial  institutions 
and  then  electronically  transfers  the  lump  sum  pay  deposits 
to  the  appropriate  financial  institutions  for  deposit  into 
each  individual's  account.  Besides  saving  postage,  the 
company  does  not  have  the  administrative  burden  associated 
with  manually  processing  the  checks.  The  electronic 
delivery  of  funds  is  more  efficient  because  there  are  no 
lost  or  misplaced  checks,  the  deposit  is  immediately 
available  for  the  employee's  use,  and  there  is  also  better 
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and  more  accurate  record  keeping  of  the  pay  process.  All 
this  is  accomplished  without  human  interaction. 

A  similar  scenario  for  EDI  was  established  between  the 
Columbus,  Ohio  based  Satellite  Glass  Corporation  and  a  San 
Antonio,  Texas  based  insurance  company,  the  United  Services 
Automobile  Association.  The  two  companies  are  using  an 
electronic  billing  system  which  has  improved  the  speed  and 
accuracy  with  which  invoices  are  processed  while  saving  an 
estimated  $25.00  per  invoice.  The  companies  process 
approximately  30,000  invoices  per  year,  for  a  total  annual 
savings  of  $750,000  (20:17,19,30). 

Similar  savings  are  possible  within  the  DOD.  For  the 
most  part,  the  DOD  is  primarily  using  EDI  for  some  ordering 
transactions  of  commercial  items  or  for  military  items 
available  through  commercial  distribution  channels.  These 
EDI  transactions  are  limited  to  a  $25,000  threshold  by 
Federal  Acquisition  Regulation  (FAR)  Part  13,  Small  Purchase 
and  Other  Simplified  Purchase  Procedures.  Contracting 
actions  of  under  $25,000  accounted  for  almost  98%  of  the  15 
million  prime  contract  awards  in  FY88  (12:IV, 1-43, 2-4) . 

With  this  number  of  contract  awards  each  year,  the  potential 
for  EDI  use  within  the  DOD  is  enormous! 

Networks 

Communication  networks  permit  the  fast  interchange  of 
information  among  the  users  of  the  network  (16:3).  Networks 
are  typically  designated  as  either  a  Local  Area  Network 
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(LAN) ,  a  Metropolitan  Area  Network  (MAN)  or  a  Wide  Area 
Network  (WAN) .  This  designation  depends  on  the  size  of  the 
network.  A  LAN  is  the  smallest  and  most  common. 

A  LAN  is  defined  as  "a  data  communications  system 
allowing  a  number  of  independent  devices  to  communicate 
directly  with  each  other,  within  a  moderately  sized 
geographic  area  over  a  physical  communications  channel  of 
moderate  data  rates,"  (16:4).  There  are  four  key  elements 
to  this  definition: 

1.  Allows  a  niimber  of  independent  devices  to 
communicate  directly  with  each  other  to  support 
station-to-station  communication. 

2.  Within  a  moderately  sized  geographical  area, 
typically  within  a  single  building  or  group  of 
buildings  but  not  normally  over  a  span  greater  than  10 
km. 

3.  Over  a  physical  communications  channel,  devices  are 
hooked  together  directly  or  via  other  communication 
medium  to  make  the  physical  connection.  The 
traditional  LAN  design  incorporates  at  least  one  of 
three  cabling  methods:  twisted  pair,  coaxial,  or 
optical  fiber. 

4.  Supports  a  moderate  data  rate,  typically  1  mbps 
(mega  byte  per  second)  to  10  mbps,  but  with  fiber  optic 
cable,  rates  up  to  100  mbps  are  possible. 

Metropolitan  Area  Networks  typically  link  sublevels  of 

an  organization  over  an  area  of  1  km  to  10  km.  A  wide  area 

network  ties  together  users  who  are  widely  separated  (over 

50  km  apart)  geographically  (16:3). 

The  use  of  organizational  wide  networks  such  as  DON  is 

becoming  more  prevalent  in  the  commercial  environment.  It 

is  estimated  that  after  1991,  60%  of  all  network  users 

expect  to  have  organization  wide  networks  in  their  firms 


(17:42).  Figure  1  below  shows  the  current  and  projected 
percentage  of  sites  with  networked  computing  systems. 
Organization  wide  networks  are  evolving  from  the 
departmental  networks  so  that  a  large  percentage  of  smaller 
networked  systems  will  be  interconnected.  The  Departmental, 
Sitewide,  and  Organizational  Networks  can  be  equated  to  the 
Local  Area,  Metropolitan  Area  and  Hide  Area  Networks 
respectively. 
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NETWORK  USAGE 
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Figure  1.  Network  Usage  (17:42) 


Electronic  Mail  f E-Mail) 

Electronic  Mail  is  defined  as  "electronic  text  message 
that  can  be  sent  to  a  remote  computer  and  stored  for 
retrieval  only  by  a  selected  addressee,"  (1:243).  This  is  a 
general  definition  that,  as  computer  technology  advances, 
will  include  an  even  larger  variety  of  message  types.  The 
two  key  concepts  of  the  definition  are  as  follows: 

Electronic  Text  Message.  Currently,  "electronic  test 
message"  refers  to  a  word  message  send  and  stored  in  a 
digital  format.  As  will  be  discussed  later  in  this  section, 
it's  possible  to  transmit  more  than  just  text  messages 
through  E-Mail  by  sending  fax  messages  through  the  computer. 


Therefore,  for  the  purpose  of  this  study,  E-Mail  will  be 
used  to  describe  both  graphics  as  well  as  text  messages. 

Sent  to  a  remote  computer  and  stored  for  retrieval  only 
bv  a  selected  addressee.  This  remote  computer  can  be  a 
central  host  computer  or  part  of  a  network  system.  The 
significance  of  this  part  of  the  definition  is  that  one  does 
not  have  to  be  immediately  present  to  receive  an  E-Mail 
message.  Similar  to  postal  delivery,  where  a  letter  remains 
in  a  mailbox  until  it  is  opened  by  the  addressee,  an 
electronic  message  is  delivered  and  stored  until  the 
addressee  retrieves  it  for  review. 

The  host  computer  system  may  be  a  commercially 
available  service  such  as  CompuServe  or  a  system  operated  by 
the  using  organization.  Within  the  Department  of  Defense 
(DOD) ,  there  is  a  network  of  hosts  that  communicate  with 
each  other  (2:16).  This  network  is  called  the  Defense  Data 
Network  (DDN)  and  is  a  data  communications  network  that 
provides  users  with  both  data  conununication  services  and  a 
medium  over  which  heterogeneous  computer  systems  can 
interoperate  (19:4).  Through  the  use  of  DDN,  DOD  personnel 
have  the  capability  to  communicate  with  any  other  member  of 
the  DOD  if  the  addressee's  username  and  its  host  computer 
designation  are  kno%m. 

There  are  benefits  and  obstacles  associated  with  using 
E-Mail.  The  next  sections  discuss  the  benefits  and  the 
problems  associated  with  the  use  of  E-Mail. 
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Benefits.  Several  benefits  associated  with  using  E- 
Mail  have  been  documented.  A  list  of  these  benefits  and  a 
brief  discussion  of  each  follow: 

Lowers  delivery  time/cost  compared  to  conventional 
mail  (5:12).  A  single  business  letter  has  been  estimated  to 
cost  between  $7.00  and  $10.00  when  one  considers  the  cost  of 
composing,  typing,  editing,  retyping,  and  mailing  the 
letter.  This  cost  increases  if  express  mail  service  is 
used.  By  comparison,  the  average  electronic  message  costs 
approximately  $3.00  (1:70). 

Reduces  the  number  of  attempts  necessary  to 
contact  someone  bv  telephone  (5:12).  As  previously  noted, 
if  an  addressee  is  not  available,  the  electronic  message  is 
stored  until  the  addressee  can  retrieve  it.  Upon  receipt, 
the  addressee  can  send  an  immediate  response  if  necessary. 

Deliveries  to  multiple  addressees  (1:71). 

Messages  can  be  simultaneously  sent  to  multiple  addresses, 
at  multiple  locations.  This  multiple  transmission  can  be 
accomplished  very  simply  through  the  use  of  predefined 
mailing  lists.  In  addition  to  not  having  to  make  photocopies 
of  a  document  to  be  mailed  out  to  several  addressees, 
messages  simply  need  to  be  coded  to  be  sent  to  multiple 
addressees. 

Eliminates  excess  paperwork  and  photocopying 
(5:12).  E-Mail  reduces  the  volume  of  paperwork  in  many 
ways.  First,  multiple  addressing  enables  the  sender  to 
transmit  one  message  to  several  addressees  at  the  same  time; 
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ellninating  the  t.eed  for  multiple  copies  or  photocopies. 
Also,  if  a  message  must  be  forwarded  to  an  additional 
addressee,  it  can  be  recalled  and  transmitted  electronically 
eliminating  the  need  to  make  additional  copies. 

Eases  problems  created  from  different  geographical 
boundaries  and  time  zones  ^5; 12).  Messages  can  be  sent  and 
received  at  any  time.  Messages  sent  at  one  time  are  stored 
until  the  recipient  is  available  to  review  them  (1:71). 

Provides  recordabilitv.  Electronically  delivered 
messages  are  received  through  a  computer  in  a  form  that  can 
be  saved  to  a  disk  or  printed.  This  is  referred  to  as 
"recordability"  (5:12).  Also,  these  messages  can  be 
reformatted  or  modified  and  then  transmitted  to  additional 
individuals. 

Limitations.  Drawbacks  or  limitations  related  to  the 
use  of  E-Mail  which  were  identified  in  the  literature  are 
discussed  below. 

Frequency  of  E-Mail  review:  If  an  individual 
fails  to  log  on  to  the  system,  the  messages  can  not  be  read. 
This  can  be  overcome  by  establishing  procedures  which 
require  individuals  to  connect  to  a  system  at  regular 
intervals  throughout  the  day.  Also,  common  protocols  can  be 
designed  within  the  computer  to  notify  the  user  that  there 
are  new  incoming  mail  messages  to  be  reviewed. 

Clutter.  Access  to  electronic  mail  systems  may 


lead  to  information  overload.  The  increased  volume  of 
Incoming  messages  may  be  difficult  to  assimilate  and  reports 


of  "garbage  electronic  nail"  are  prevalent  (18:96). 

Computer  firms  are  currently  working  on  filters  that  sort 
incoming  messages  based  on  subject  title  or  originator. 
Additional  protocols  within  the  filter  will  allow  the 
message  to  be  scanned  for  key  words  to  aid  in  the  sorting 
process.  As  more  LANS  are  interconnected  the  amount  of  E- 
mail  traffic  will  increase.  Devices  that  reduce  mailbox 
clutter  will  become  more  valuable  (3:286). 

Original  copies  A  fundamental  issue  concerning 
electronic  delivery  and  computer  storage  is  whether 
electronic  transaction  forms  constitute  an  agreement  that  is 
legally  sufficient  to  bind  parties  (12:4-1).  Drake 
elaborate:  "In  commercial  practice,  EDI  participants  agree 
to  be  bound  by  electronic  transactions  in  accordance  with  a 
trading  partner  agreement,"  (12:4-1).  Legal  acceptability 
of  electronic  medium  transmission  and  storage  is  still  under 
review. 

Concerning  optical  (electronic)  storage,  one  firm,  the 
USAA  is  relying  on  the  best  available  evidence  rule  when 
considering  whether  to  destroy  original  documents  after  the 
document  has  been  scanned  and  electronically  stored.  The 
best  available  evidence  rule  states  that  if  the  original  is 
available,  it  must  be  entered  as  evidence  in  the  case  of 
legal  dispute.  If  the  original  is  destroyed,  then  the  image 
becomes  the  best  evidence  available  and  the  official 
business  record  (15:81). 
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For  Electronic  Data  Delivery  to  become  an  acceptable 
mode  of  delivery,  appropriate  Federal  Procurement 
regulations  need  to  recognize  EDI  relationships  and  bind  the 
contracting  pairties  when  certain  electronic  conventions  are 
met  (12:4-1). 

F&QsipU?_M?.ghin^.s 

Fax  machines  have  been  in  use  since  the  1940s.  Prior 
to  1967,  most  FAX  machines  were  produced  by  Bell  Telephone. 
In  1967,  a  ruling  by  the  Federal  Cosununications  Commission 
(FCC)  allowed  for  other  than  AT&T  equipment  to  be  connected 
to  Bell  Telephone  lines.  This  ruling  made  competition 
possible  in  several  markets  to  include  the  FAX  machine.  FAX 
machine  usage  continues  to  increase  because  the  FAX  is  easy 
to  use,  fast,  inexpensive,  flexible  and  well  standardized. 
(1:169-170) . 

In  addition  to  the  conventional  Fax  machine,  PC-Fax 
systems  are  also  gaining  popularity.  These  systems  utilize 
the  Personal  Computer's  (PC's)  capabilities  by  adding  a  "PC- 
Fax"  board  to  one  of  the  expansion  slots  on  a  PC.  The  main 
difference  between  a  conventional  Fax  and  a  PC-Fax  is  that 
with  a  conventional  Fax  the  document  must  be  fed  into  a 
machine  and  the  phone  number  dialed  before  the  document  is 
sent  (1:178).  The  PC-Fax  simplifies  the  process  by  allowing 
the  Fax  to  be  sent  directly  from  the  computer.  An 
additional  advantage  is  that  a  printout  is  not  necessary  for 
transmission.  Also,  like  E-Mail,  a  message  can  be  sent  to 
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multiple  addresses,  and  stored  as  a  electronic  file  or 
hardcopy . 

Despite  these  advantages,  limitations  also  exist. 
Albright  lists  seven  disadvantages  of  using  a  PC-Fax  system. 
For  instance,  a  PC~Fax  system  requires  a  scanner  to  send 
pre-existing  documents.  This  requires  extra  hardware  and 
software  to  send  a  document  into  the  computer  for 
transmissions.  As  technology  advances,  this  will  be  less  of 
a  limitation.  Further,  if  most  of  the  transmissions  are  of 
materials  (letter,  forms,  etc.)  produced  on  a  computer,  a 
scanner  may  not  be  required  (1:182).  Also,  a  PC-Fax  system 
also  requires  additional  hard  disk  space.  This  hard  disk 
space  is  necessary  because  fax  output  storage  may  occupy  up 
to  ten  times  the  required  storage  as  a  similar  ASCII  output 
file. 

For  the  price  of  a  few  stand  alone  fax  machines  or  PC- 
fax  boards,  LAN  managers  can  give  all  network  users  access 
to  a  fax  server  which  allows  fax  service  to  be  shared  across 
the  network.  There  are  trade-offs  in  using  a  fax  server 
versus  a  stand-alone  fax.  Fax  servers  may  lead  to 
processing  delays  and  fax  servers  have  trouble  routing 
incoming  faxes.  However,  it  must  be  noted  that  the  user  may 
wait  in  line  to  send  a  fax  in  the  conventional  way  over  a 
fax  machine.  When  using  a  fax  server  the  wait  is  absorbed 
by  the  computer  while  the  user  is  free  to  do  something  else. 
Routing  incoming  faxes  could  be  accomplished  by  connecting 
all  faxes  to  a  central  destination  (administrative  position) 
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and  allowing  that  user  to  do  the  routing.  Ideally,  to  limit 
disk  storage  usage,  all  fax  processing  and  conversion  should 
be  performed  at  the  server.  This  prevents  an  accumulation 
of  disk  storage  space  because  ASCII  file  format  does  not 
require  as  much  space  as  the  standard  fax  output  format. 
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Ill,  Methodology 

This  chapter  describes  the  methodology  used  to  complete 
the  research  objectives  and  to  answer  the  research  questions 
listed  in  chapter  I  of  this  thesis.  The  survey 
justification,  the  population  from  which  the  data  was 
collected,  the  survey  instruments  used  to  collect  the  data, 
and  data  collection  and  analysis  are  discussed. 

Justification  of  Survey 

The  information  and  data  required  to  complete  this 
study  was  not  available,  therefore  it  was  necessary  to 
collect  primary  data.  Several  methods  of  data  collection 
were  considered  to  include:  observation,  experimentation, 
personal  interviews,  telephone  interviews,  personally 
administered  surveys,  and  mail  surveys.  A  combination  of 
mail  surveys  and  telephone  interviews  was  selected  for 
several  reasons.  These  factors  include:  the  size  of  the 
populations,  time  constraints,  the  nature  of  the 
information,  and  most  importantly,  funding  constraints. 

Hail  surveys  are  usually  less  costly,  allow  contact 
with  individuals  who  might  otherwise  be  inaccessible,  allow 
respondents  to  take  more  time  to  complete  the  survey,  and 
provide  anonymity  (13:172).  The  major  weaknesses  of  the 
mail  survey  are  nonresponsiveness  and  incompleteness  due  to 
excessive  survey  length  (13:172).  The  response  rate  was  not 
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a  problem  because  the  surveys  were  hand-delivered  to  each 
duty  section  for  individual  distribution. 

Telephone  interviews  are  inexpensive  to  complete 
especially  if  "callbacks"  are  necessary  and  if  respondents 
are  widely  scattered.  Also,  the  telephone  interview  allows 
for  the  study  to  be  completed  more  quickly  (13:170).  The 
major  weaknesses  of  the  telephone  interview  are  that  the 
individual  must  be  available  by  phone  and  that  there  are 
limits  to  the  complexity  of  the  questioning  (13:171).  The 
questioning  was  simple  and  because  those  interviewed  were 
government  employees,  they  had  access  to  at  least  one 
telephone.  The  data  collection  strategy  anticipated  that 
callbacks  would  be  required. 

Mail  Survey.  The  population  of  interest  for  the  mail 
survey  portion  of  this  study  was  Air  Force  acquisition 
professionals  who  are  currently  involved  in  data  management 
activities  to  include  data  managers,  data  entry  personnel, 
configuration  managers,  and  program  managers,  as  well  as 
project  engineers  and  logistics  managers.  There  is  no 
specific  duty  title  or  specialty  code  for  data  management 
positions;  therefore,  the  sample  for  this  study  was  taken 
from  program  offices  at  Aeronautical  Systems  Division, 
Wright-Patterson  AFB  and  Human  Systems  Division,  Brooks  AFB 
respectively.  These  samples  include  offices  from  two  of  the 
four  Air  Force  acquisition  product  divisions. 
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There  were  twenty  data  management  offices  identified  at 
Wright-Patterson  AFB  and  one  data  management  office  at 
Brooks  AFB.  The  number  of  individuals  assigned  to  each 
office  was  determined  by  contacting  each  office  by  phone. 

It  was  determined  that  there  were  approximately  195 
individuals  assigned  to  the  data/configuration  management 
offices.  This  number  included  contractor  support  personnel 
working  in  the  area  of  data  and  configuration  management. 

Telephone  Interview.  The  population  of  interest  for 
the  telephone  interview  portion  of  this  study  consisted  of 
contracting  personnel  responsible  for  contracts  awarded 
within  ASD/PK  from  1  October  1989  through  31  May  1991.  This 
population  was  chosen  because  contracts  awarded  during  this 
time  had  the  opportunity  to  take  advantage  of  advances  in 
computer  technology  and  accept  the  electronic  delivery  of 
data. 

There  were  a  total  of  197  contracts  awarded  during  this 
time:  118  from  1  October  1989  through  30  September  1990  and 
79  from  1  October  1990  to  31  May  1991.  Modifications  to  and 
options  exercised  on  existing  contracts  during  this  period 
were  not  included  in  the  sample  population. 

Survey  Instruments 

There  were  two  survey  instruments  specifically 
developed  for  this  effort.  One  instrument  was  a 
questionnaire  used  in  the  mail  survey,  and  the  second 
instrument  was  an  interview  guide  used  to  support  the 
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telephone  interviews.  The  questionnaire  and  the  interview 
guide  are  provided  in  Appendices  A  and  B,  respectively. 


Approval .  The  HQ  AFMPC/DPMYUS ,  Randolph  AFB,  TX 
granted  the  survey  and  interview  approval,  assigning  the 
USAF  Suirvey  Control  Nxunber  (SCN)  91-49.  This  SCN  expired  on 
30  September  1991. 

Data  Analysis  and  Reduction 

The  data  obtained  through  the  survey  questionnaires  and 
telephone  interviews  was  analyzed  to  answer  the 
investigative  questions  outlined  in  Chapter  1.  These 
investigative  questions  were  addressed  by  either  the  survey 
questionnaire,  telephone  interviews  or  both.  Table  1 
provides  a  correlation  between  the  investigative  question, 
survey  question,  interview  question  and  the  method  of 
analysis.  The  following  sections  provide  the  methodology 
used  to  answer  each  of  the  investigative  questions,  in  this 
study . 


Table  1 

Methodology  Correlation  Matrix 


Investigative 

Question 

Survey 

Question 

Interview 

Question 

Method  of 

Analysis 

1 

7 

— 

Binomial  Test 

2 

5 

— 

Descriptive 

Statistics 

8,9,10,&11 

— 

Binomial  Test 
Descriptive 

4 

14,15 

Descriptive 

5 

16 

2 

Binomial  Test 

6 

20 

3 

Descriptive 

7 

17 

10 

Descriptive 

8 

18,19 

6,7 

Descriptive 

9 

5 

Descriptive 

10 

21 

9,11 

Descriptive 

Investigative  Question  Number  !♦  Are  program  office 
personnel  aware  of  the  DOD  requirements  concerning  digital 
data  delivery? 

Question  number  7  of  the  survey  questionnaire  (Appendix 
A)  was  developed  to  determine  whether  program  office 
personnel  were  able  to  identify  the  correct  DODI  5000.2 
direction  concerning  the  digital  delivery  of  data. 

Responses  to  this  question  were  categorized  as  either 
correct  or  incorrect.  These  categories  were  totaled  for  the 
sample  at  large  and  specifically  for  data  managers. 
Statistical  tests  were  run  on  this  data  to  determine  if 
acquisition  personnel  and  specifically  data  personnel  knew 
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the  correct  DOD  policy.  A  binomial  test  was  used  because 
the  data  satisfied  the  requirements  of  independent  Bernoulli 
trials.  These  requirements  are  as  follows: 

1.  The  experiment  consists  of  a  sequence  of  n  trials, 
where  n  is  fixed  in  advance  of  the  experiment. 

2.  The  trials  are  identical,  and  each  trial  can 
result  in  one  of  the  same  two  possible  outcomes, 
which  are  denoted  as  either  a  success  or  failure. 

3.  The  trials  are  independent,  so  that  the  outcome  on 
any  particular  trial  does  not  influence  the 
outcome  of  any  other  trial. 

4.  The  probability  of  success  is  constant  from  trial 
to  trial;  we  denote  this  probability  by  "p" 

(11:98) . 

Responses  to  this  question  could  be  categorized  as 
"successes"  (answer  "d"  selected)  or  as  "failures"  (answer 
"d"  not  chosen) .  The  probability  of  success  for  each  trial 
was  assumed  to  be  constant  as  there  was  no  reason  to  assume 
otherwise.  The  number  of  trials  is  fixed  for  the  total 
number  of  questionnaires  and  finally,  each  of  the  n  trials 
was  independent. 

Investigative  Question  Number  2.  Are  the  necessary  program 
office  personnel  trained  in  data  management? 

This  question  was  asked  to  determine  whether  or  not 
acquisition  personnel  are  trained  in  data  management 
procedures  and  techniques.  The  two  courses  in  the  area  of 
data  management  taught  at  the  Air  Force  Institute  of 
Technology  (AFIT)  were  provided  as  two  of  the  possible 
responses  along  with  the  open  ended  third  selection  of 
"other".  This  was  left  open  because  there  are  a  number  of 
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data  related  courses  taught  by  civilian  institutions  as  well 
as  other  AFIT  courses  that  nay  have  blocks  of  information 
dedicated  to  instruction  in  the  area  of  contractor  data. 

The  responses  to  this  question  were  compiled  by 
functional  area  and  as  a  total  sample. 

Investigative  Question  Numbers  3  and  4.  Are  the  necessary 
program  office  personnel  trained  in  the  use  of  Electronic- 
Mail  (E-Mail)?  Are  personnel  trained  in  the  use  of 
networked  computer  systems?  (Local  area  networks?) 

Survey  c[uestions  8,  9,  10,  11,  12,  13,  14,  and  15  were 
asked  to  examine  E-Mail  and  LAN  training  and  use.  It  is 
important  to  know  if  individuals  have  access  to  a  computer. 
If  access  to  a  computer  and  a  phone  line  are  available,  then 
it  is  possible  to  receive  data  electronically.  Electronic 
mail,  although  similar  on  most  systems,  has  some 
peculiarities  unique  to  an  individual  system.  Therefore, 
some  training  in  the  use  of  E-mail  may  be  required. 

Freq[uently,  copies  of  a  particular  data  item  are 
required  by  individuals  within  the  same  functional  entity. 
It's  not  uncommon  for  these  functional  entities  or  divisions 
to  be  linked  together  through  a  LAN.  Therefore,  the 
capability  to  pass  information  electronically  over  the  same 
system  or  to  have  the  capability  for  these  individuals  to 
have  access  to  the  same  document  would  greatly  enhance  the 
operation  of  the  organization.  This  would  be  particularly 
true  when  review  of  a  document  has  resulted  in  a 
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consolidated  comment  list  which  must  be  transmitted  to  the 
contractor. 

Questions  14  and  15  of  the  survey  questionnaire  were 
asked  to  ascertain  if  individuals  are  part  of  a  Zxjcal  Area 
Network  and  if  they  have  been  trained  in  the  use  of  Local 
Area  Networks. 

Investigative  Question  Number  S.  Is  electronic  data 
delivery  being  applied  to  current  Air  Force  acquisition 
programs  within  ASD? 

Survey  question  number  16  and  interview  question  nximber 
2  were  asked  to  answer  this  investigative  question.  Survey 
question  number  16  did  not  limit  the  respondent  to 
discussing  a  specific  contract  but  allowed  the  respondent  to 
describe  any  contract  that  he  or  she  may  have  been  working. 
Although  responses  were  not  unique  to  particular  contracts, 
the  responses  to  this  question  can  be  totaled  to  represent 
the  number  of  individuals  currently  involved  in  the 
electronic  delivery  of  data.  Interview  question  number  2 
was  asked  to  obtain  a  response  concerning  a  particular 
contract  and  provides  more  detailed  information  regarding 
the  percentage  of  current  contracts  that  are  using 
electronic  delivery  of  data.  For  both  of  these  questions, 
analysis  was  conducted  using  the  binomial  test. 

Investigative  Question  Number  10.  Is  other  contract 
correspondence  electronically  transferred  between  the 
government  and  contractor? 
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Survey  question  number  21  and  interview  question 
numbers  9  and  11  were  asked  to  answer  this  question. 
Interview  questions  9  and  11  are  the  same  question  asked  in 
different  situations  in  the  interview  process  depending  on 
whether  or  not  a  specific  contract  requires  the  electronic 
delivery  of  contractor  data.  Both  of  these  questions  were 
not  asked  of  the  same  individuals,  therefore,  it  was 
possible  to  combine  these  responses  for  stuzimation  and 
analysis  without  duplicating  responses. 

Limitations 

The  samples  for  both  the  mail  survey  and  interview  were 
taken  from  configuration/data  management  offices  at  ASD  and 
HSD  respectively.  As  previously  mentioned,  the  study  was 
further  limited  because  the  list  of  contracts  used  as  a 
sample  for  this  study  did  not  include  modifications  to 
existing  contracts  nor  options  exercised  between  1  Oct  89 
and  31  May  91.  This  limit  is  significant  because  it  may  be 
during  these  later  modifications  and  options  that  the 
program  offices  and  the  contractor  implemented  an  electronic 
delivery  system. 

The  following  chapter  provides  an  analysis  of  the  data 
collected  through  the  mail  surveys  and  interview  process  for 
the  specific  questions  discussed  in  this  chapter. 
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This  chapter  provides  the  results  of  the  analysis 
performed  on  the  data  collected  using  the  survey 
questionnaire  and  interview  guide.  A  descriptive  summary  of 
results  is  provided  along  with  the  results  of  any 
statistical  tests  performed  on  the  responses. 

Survey  Response  Summary 

A  total  of  78  questionnaires  were  returned  from  the 
original  217  survey  packages  distributed.  These  78 
questionnaires  represent  a  response  rate  of  35.9  percent. 
Typically  response  rates  of  30  percent  or  more  are 
considered  adequate  for  mail  surveys  (13:172).  A  breakdown 
of  respondents  by  functional  category  is  provided  in  Table 
2. 

Interview  Summary 

Sixty^two  individuals  were  identified  as  suitable 
interview  candidates.  Forty-five  persons  were  successfully 
interviewed,  and  these  individuals  were  responsible  for  109 
of  the  197  contracts  awarded  between  1  Oct  1989  and  31  May 
1991. 

Knowledge  of  Requirements 

Only  23  of  the  78  respondents  knew  that  contract 
deliverable  data  will  be  prepared  and  used  in  digital  form 
unless  it  is  not  cost  effective  for  the  government.  The 
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binomial  test  provides  the  following  95%  confidence  interval 
on  the  true  value  of  p:.20  <  p  <  .42  (4:445).  Therefore, 
with  95%  probability  that  the  true  proportion  of  individuals 
that  know  the  DODI  requirements  is  between  20  percent  and  42 
percent. 


TABLE  2 

Functional  Category  of  Respondents 


Functional  Category 

Number  of 
Respondents 

Percentage  of  Total 

Data  Management 

39 

50.0 

Configuration 

Management 

30 

38.5 

Other: 

9 

11.5 

N/A 

2 

2.6 

PM 

3 

3.8 

ENG 

3 

3.8 

Log 

1 

1.3 

Total 

78 

100.00 

Data  Management  Training 

This  research  investigated  the  training  received  by 
individuals  currently  involved  with  data  management 
activities.  Data  management  training  is  susunarized  in  Table 
3  and  Figure  2. 

As  noted  in  Table  3,  59%  of  all  individuals  surveyed 
have  taken  SYS  370,  Defense  Data  Management.  As  expected, 
the  greatest  percentage  (27  of  the  47  individuals  who  have 
taken  SYS  370)  were  data  managers.  Data  and  Configuration 
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Table  3 


Data  Management  Training  Summary 


Figure  2.  Data  Management  Training 


Management  personnel  accounted  for  all  those 
individuals  who  have  received  Data  Management  Training.  The 
total  nximber  of  data  management  personnel  who  have  been 
trained  through  formal  classroom  work  in  data  management  is 
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31  out  of  a  possible  39  for  a  percentage  of  approximately 
80%. 

Availability  of  Computer  Resources 

This  research  investigated  the  computer  resources 
available  to  acquisition  personnel.  This  included  the 
availability  of  computer  terminals  or  PCs  as-well-as  access 
to  Local  Area  Networks.  Only  one  of  the  respondents  did  not 
have  either  a  personal  computer  on  their  desk  or  did  not 
share  a  personal  computer  with  others  in  the  duty  area. 

In  addition  to  personal  computers,  16  of  the  78 
respondents  had  access  to  a  VAX  terminal  in  their  immediate 
duty  area.  This  value  may  be  misleading  because  the  term 
"VAX”  is  ambiguous.  The  intent  of  this  response  item  was  to 
determine  how  many  individuals  had  direct  access  to  the  base 
E>mail  system  and  subsequently  to  the  Defense  Data  Network 
to  transfer  E-mail  throughout  the  DOD.  However,  one  does 
not  need  a  dedicated  VAX  terminal  in  order  to  have  access  to 
the  DDN,  one  only  needs  the  proper  software  to  allow 
communications  from  a  personal  computer  into  the  VAX  system. 
Therefore,  some  individuals  may  have  indicated  that  they 
have  access  to  a  VAX  terminal  when  in  fact  they  were  using 
an  emulation  program  to  communicate  to  the  VAX  from  a 
personal  computer. 

A  summary  of  those  individuals  that  have  access  to  a 
Local  Area  Network  is  provided  in  Figure  3.  Of  the  73 
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individuals  that  responded  to  this  question,  56  (or  76.7 
percent)  indicated  that  they  have  access  to  a  LAN.  Only 


LAN  ACCESS 

Survey  Responses 


Figure  3.  LAN  Access 


four  (5.58  percent)  of  the  individuals  indicated  that  they 
did  not  have  access  to  a  LAN.  A  total  of  15  (20.5  percent) 
of  the  individuals  either  did  not  know  if  they  were 
connected  to  a  local  area  network  or  did  not  know  what  a 
local  area  network  was. 

The  binomial  test  indicates  that  the  95%  confidence 
interval  on  the  true  value  of  p  (probability  that  an 
individual  was  connected  to  a  LAN  given  a  random  sample  was 
taken)  is  0.82  <  p  <  0.96.  In  other  words,  with  a  95% 
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confidence  the  true  mean  for  the  number  of  individuals  that 
are  connected  to  a  LAN  is  between  82  and  96  percent. 


This  research  investigated  the  training  acquisition 
personnel  received  in  the  use  of  Electronic  Mail  (E>Mail) 
and  the  level  of  usage  of  E-mail  by  acquisition  personnel. 

A  summary  of  E-Mail  training  is  provided  in  Figure  4,  a 
summary  of  E-mail  usage  is  provided  in  Figure  5,  and 
comparison  of  training  received  and  daily  usage  of  E-mail  is 
illustrated  in  Figure  6. 

Approximately  41%  of  the  respondents  received  no  fonnal 
training  in  the  use  of  E-mail.  However,  almost  70%  of  the 
individuals  use  E-mail  at  least  once  a  week  and  a  majority 
(59.0%)  of  the  individuals  use  E-mail  at  least  once  a  day. 


Figure  4.  E-Mail  Training 


Figure  5.  E'-Mail  Usage 


As  noted  in  chapter  II,  the  value  of  E-mail  is  enhanced  the 
more  it  is  used.  A  further  analysis  of  the  data  reveals 
that  18  (56.3%)  of  32  individuals  who  received  no  training 
used  E-mail  at  least  once  a  day  while  10  (31.3%)  of  these 
individuals  use  E-mail  once  a  month  or  less.  Also,  12 
(50%)  of  the  24  individual  who  had  only  1-2  hours  of  E-mail 
training  use  E-mail  at  least  once  a  day. 

For  those  individuals  that  received  three  or  more  hours 
of  training,  the  E-mail  usage  rate  of  at  least  once-a-day 
was  approximately  72%.  Therefore,  the  data  suggests  that  if 
individuals  are  provided  with  training,  they  would  use  E- 
mall  on  a  more  regular  basis  than  those  with  no  training. 


Figure  6.  E-Mail  Daily  Use 


Of  the  78  survey  respondents,  68  respondents  knew 
their  user  name,  6  did  not  know  their  user  name,  and  4 
individuals  did  not  respond.  A  95%  confidence  interval  on 
the  true  value  of  p  (probability  that  an  individual  would 
know  their  e-mail  user-names  given  a  random  sample  was 
taken)  was  obtained  from  a  binomial  confidence  limit  chart 
and  is  as  follows:  0.83  <  p  <  0.97.  In  other  words,  there 
is  a  95%  probability  that  the  true  proportion  of  individuals 
who  know  their  own  E-mail  user  name  is  between  83  and  97 
percent . 
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IAN  Training 

This  research  investigated  how  much  IAN  training  was 
received  by  acquisition  personnel.  The  results  are 
illustrated  in  Figure  7.  Of  the  76  individuals  who  answered 
this  question,  40  (52.6  percent  of  the  total)  have  received 


LAN  TRAINING 

Survey  Responses 


NO  TRAINING  (526%) 


<  4  HOURS  (27.6%) 


>  16  HOURS  (5.3%) 
8-16  HOURS  (5.3%) 
4-8  HOURS  (9.2%) 


Figure  7.  IAN  Training 

no  training.  An  additional  27.6%  have  received  less  than  4 
hours  of  IAN  training. 


Application  of  Electronic  Delivery  to  CDRL  Item  Data 

This  research  explored  the  extent  that  electronic 
delivery  of  contractor  data  is  being  used  on  current 
contracts.  Of  the  109  interview  responses,  7  reported 
electronic  delivery  of  data  was  required.  Of  the  78  survey 
responses,  11  individuals  indicated  they  were  working  on 
programs  that  required  the  electronic  delivery  of  contractor 
data . 
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A  95%  confidence  interval  on  the  true  value  of  c 
(probability  that  electronic  delivery  of  data  is  required  on 
a  prograin/contract)  was  performed  for  the  responses  to  both 
the  interview  and  survey  responses.  The  following  95% 
confidence  intervals  were  determined:  0.04  <  p  <  0.13  for 
the  interview  process,  and  0.09  <  p  <  0.24  for  the  survey 
responses.  With  95%  confidence  it  could  be  expected  that  no 
more  than  13%  of  contracts  require  electronic  data  delivery 
and  no  more  than  24%  of  the  individuals  are  working  programs 
that  require  electronic  data  delivery. 

There  could  be  several  reasons  for  the  large  difference 
between  the  expected  values  for  the  mean  of  survey  responses 
and  the  interview  responses.  One  reason  could  be  that 
several  respondents  are  assigned  to  the  same  program  office, 
therefore,  the  survey  responses  may  contain  duplicates. 
Another  reason  is  that,  as  mentioned  in  chapter  III,  the 
interview  process  responses  were  only  applicable  to 
contracts  awarded  from  1  October  1989  through  31  May  91. 

The  survey  responses  were  applicable  to  any  current 
contracts  to  include  modifications  and  options.  Also,  as 
noted  above,  although  the  interview  respondents  and  the  mail 
survey  participants  were  taken  from  the  same  population 
(acquisition  professionals)  they  were  taken  from  different 
samples . 


This  study  also  reviewed  which  CDRL  items  were  being 


delivered  through  electronic  means.  Two  of  the  seven 
respondents  indicated  that  the  contracts  they  are 
responsible  for  require  all  CDRL  items  be  delivered 
electronically.  For  the  remaining  five  contracts  the  most 
common  items  to  be  delivered  were  software  related:  the 
software  code  and  software  configuration  items,  and  also  the 
associated  documentation.  The  following  types  of  data  are 
received  electronically  on  at  least  one  contract: 
provisioning  data,  test  specifications,  contract  schedules, 
drawings  and  a  post>production  support  analysis  list. 

Benefits  of  Using  Electronic _Deliverv  of  Data  (EDD) 

Interview  responses  and  survey  question  responses  were 
consistent  with  each  other  and  were  identified  benefits 
similar  to  those  revealed  in  the  literature.  Two  of  the 
seven  contracts  using  electronic  delivery  of  data  were 
awarded  very  recently,  consec[uently  data  has  not  yet  been 
delivered.  On  four  of  the  remaining  five  contracts,  the  use 
of  electronic  delivery  for  data  has  proved  beneficial  in 
saving  time.  Respondent  also  indicate  E-mail  requires  less 
paperwork,  provides  more  flexibility  to  manipulate 
contractor  data  for  inclusion  into  larger  reports,  and 
provides  the  ability  to  more  easily  consolidate  government 
comments  for  response  to  the  contractor. 


The  results  of  the  survey  indicate  that  the  speed  of 
data  delivery  and  the  speed  at  which  the  government  can 
respond  to  the  contractor  are  the  primary  benefits.  Also, 
the  ability  to  distribute  copies  to  others  was  mentioned 
twice  (in  one  instance  docviments  were  retransmitted  via  DDN 
to  other  organizations) .  Additional  benefits  included  more 
flexibility,  ease  of  filing,  less  paperwork,  and  the  ability 
to  import  part  or  all  of  one  document  into  other  documents. 

Limitations  of  Using  Electronic  Delivery  of  Data  fEDD^ 

The  results  indicate  the  most  common  limitations  are 
incompatibility  with  contractor  systems  and  lack  of  guidance 
concerning  ways  to  obtain  the  electronic  delivery  of  data. 
One  interview  respondent  indicated  technical  "glitches” 
sometimes  cause  the  government's  computers  to  go  down; 
however,  even  with  this  loss  of  time,  delivery  is  still 
faster  than  using  regular  mail.  It  was  also  mentioned  that 
a  hard-copy  is  still  necessary  for  the  file,  and  it  is 
easier  to  review  a  document  if  there  is  a  hardcopy.  One 
respondent  noted  that  there  are  problems  maintaining  a 
docximent's  configuration  because  documents  are  easy  to 
change . 

Barriers  to  Electronic  Delivery  of  Data  (EDD) 

This  research  investigated  the  most  common  reasons 
electronic  delivery  of  data  is  not  being  used.  The  survey 
and  interview  responses  were  categorized  and  Figures  8  and  9 
show  a  comparison  of  the  most  common  responses. 
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BARRIERS  TO  EDD 

interview  Responses 


OTHER  (a8%)- 


UNTRAINED  (20.6%)- 


NOT  REQUIRED  (11.8%)-^^^^ 
FEWCDRLs(8.8%)- 


-UNCERTAIN  (24.5%) 


-SMALL  BUSINESS  (25.5%) 


Figure  8.  Barriers  to  EDD:  Interview  Responses 


The  most  common  response  for  each  (question  was  that 
individuals  did  not  know  why  electronic  delivery  of  data  was 
not  used.  On  26  of  the  109  contracts,  electronic  delivery 
of  data  was  not  used  because  the  contracts  were  either  small 
business  research  and  development  (R  &  D)  contracts  with  few 
data  items  or  were  with  a  small  business  that  did  not  have 
the  capability  to  delivery  data  electronically. 

The  responsible  individuals  for  21  (20.6%)  of  the  109 
contracts  indicated  that  electronic  delivexr  was  not  used 
because  either  they  did  not  know  the  capability  existed  for 
the  government  to  receive  data  electronically  or  government 
personnel  were  not  trained  to  accept  the  data 
electronically.  The  responsible  individuals  for  12  (11.8%) 
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Figure  9.  Barriers  to  EDD:  Survey  Responses 

of  the  contracts  stated  that  electronic  delivery  of  data  was 
not  required  (not  put  on  contract  by  program,  technical,  or 
data  management  personnel)  or  was  not  considered  at  the  time 
of  contract  award. 

The  results  from  the  survey  respondents  indicate  that 
many  (22.4%)  are  working  on  older  programs  which  did  not 
require  the  electronic  delivery  of  data.  Another  9  (13.4%) 
responded  that  they  are  not  currently  using  electronic 
delivery,  but  they  are  planning  to  use  it  on  future 
procurements.  Four  individuals  indicated  that  they  needed 
guidance  on  how  to  write  the  requirements  into  a  contract 
and  how  to  receive  the  data.  Also,  another  four  other 
individuals  felt  that  it  was  not  cost  effective  to  require 


the  electronic  delivery  of  data.  Six  individuals  stated 
that  they  did  not  have  the  resources  or  capability  to 
receive  data  electronically. 


This  research  investigated  the  methods  of  electronic 
delivery  used  on  those  contracts  that  required  the 
electronic  delivery  of  data.  The  results  are  depicted  in 
figure  10  below. 


METHODS  OF 

ELECTRONIC  DELIVERY 


DEDICATED  TERMINAL  (14.3%) 


DDN  (14.3%) 


MAQ  TAPE/FLOPPY  DISK  (71.4%) 


Figure  10.  Methods  of  Electronic  Delivery 


These  results  are  deceiving  in  that  although  magnetic  tape 
or  floppy  disk  appears  to  be  used  more,  they  are  not  used 
for  all  data  item  deliveries.  The  five  contracts  that  are 
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using  magnetic  tape  or  floppy  disk  as  the  delivery  methods 
are  those  contracts  that  are  receiving  only  a  few  data  items 
electronically.  The  other  two  contracts  rec[uire  electronic 
delivery  of  all  contract  deliverables  and  are  using  either  a 
dedicated  circuit  or  file  transfer  through  the  DON  as  the 
delivery  method. 

Application  of  Electronic  Delivery  to  non-CDRL  Item  Data 

The  results  regarding  electronic  delivery  of  non-CORL 
items  are  interesting  but  not  surprising.  On  104  of  109 
contracts,  electronic  delivery  of  non-CDRL  item  data  is 
being  used.  The  methods  used  for  delivery  were  either  the 
facsimile  machine  (Fax) ,  disk  file  transfer,  electronic  mail 
(over  either  DDN  or  dedicated  circuit) ,  or  a  combination  of 
methods.  The  results  were  not  surprising  because  the  use  of 
the  Fax  was  included  as  an  option  for  electronic  delivery. 
The  Fax  machine  was  used  on  104  of  the  contracts.  Floppy 
Disk/Magnetic  tape  was  used  on  9  contracts,  and  E-Mail  was 
used  on  2  contracts. 

The  primary  purpose  for  using  the  Fax  machine  was  to 
provide  advance  copies  of  information.  Examples  of  items 
transmitted  via  the  fax  include:  incoming  data  deliverables 
from  the  contractor,  government  response  letters  to  the 
contractor,  or  copies  of  contract  modifications  or  general 
correspondence  such  as  meeting  minutes  and  agendas.  In 
almost  all  cases  the  respondent  stated  that  fax  transmittals 
were  followed  with  a  mailed  hardcopy.  The  reason  a  hardcopy 
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(with  signatures)  is  needed  is  that  an  original  copy  of  a 
document  has  to  be  kept  in  the  contract  file.  Also,  several 
individuals  mentioned  that  the  paper  used  in  Fax  machines 
tends  to  yellow  quickly,  therefore,  some  offices  make  copies 
of  incoming  Fax's  to  use  as  temporary  copies  in  the  official 
files  until  the  official  copy  is  delivered.  The  process  of 
using  the  Fax  provides  information  faster  than  using  the 
mail;  however,  there  is  an  increase  in  the  amount  of  paper 
generated  and  an  "official"  copy  is  still  mailed. 

The  use  of  flexible  floppy  disk  has  also  been  used  to 
receive  pricing  data  associated  with  contract  proposals.  In 
this  way,  the  pricing  information  is  downloaded  and 
reformatted  for  analysis  and  inclusion  into  other  reports. 

On  one  contract,  draft  specifications  are  received  on  floppy 
disk.  Corrections  and  comments  are  made  to  the 
specification  as  necessary  and  the  specification  returned  to 
the  contractor  for  formal  submittal. 

E-mail  was  mentioned  by  two  respondents.  E-mail  is 
available  for  use  on  one  contract  but  neither  the  government 
nor  the  contractor  are  taking  advantage  of  this  capability. 
On  a  second  contract,  everything  is  sent  over  a  dedicated 
circuit  and  the  government  also  has  limited  access  to  view 
the  contractor's  data.  This  type  of  access  is  similar  to 
the  CITIS  discussed  in  Chapter  II. 

Conclusions  based  on  the  results  and  analysis  provided 
in  this  chapter  are  presented  in  chapter  V.  Also,  with 
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V,  Conclusions  and  Recommendations 

Conclusions 

The  objective  of  this  research  was  to  determine  the 
extent  that  electronic  delivery  of  contractor  data  is  being 
used  on  contracts  with  the  ASD  and  the  HSD.  This  study 
addressed  Contract  Data  Requirements  List  (CDRL)  items  as 
well  as  non-CDRL  items.  To  meet  this  objective,  the 
following  four  subobjectives  were  developed. 

Subobiective  1.  Analyse  the  extent  to  which  electronic  data 
delivery  is  being  used  by  program  offices  within  X8D  and 
HSD. 

This  study  has  revealed  that  electronic  delivery  of 
data  is  not  the  primary  method  for  delivery  of  contracting 
data.  The  primary  method  is  still  hardcopy.  Less  than 
seven  percent  of  the  contracts  reviewed  are  using  some  form 
of  electronic  delivery  for  contract  deliverables.  These 
percentages  appear  low;  however,  it  wasn't  until  February 
1991  that  electronic  delivery  became  a  policy  of  the 
Department  of  Defense  when  DODI  5000.2  was  approved. 

Results  of  the  mail  survey  responses  indicate  that  although 
many  programs  are  not  using  electronic  delivery  now, 
additional  programs  are  expected  to  utilize  electronic 
delivery  on  future  contracts. 
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Informal  usage  of  electronic  transfer  of  data  is  being 
used  on  almost  every  contract.  The  primary  method  is  the 
facsimile  (fax)  machine  which  was  used  on  almost  95%  of  the 
contracts  to  pass  advance  copies  of  documents  to  and  from 
the  contractor.  Consistent  with  the  literature,  the  main 
reason  the  facsimile  was  not  used  for  formal  deliveries  was 
because  it  was  felt  by  the  respondents  that  is  necessary  to 
have  an  original  copy  of  the  deliverable  in  the  official 
file. 

There  are  several  phases  that  businesses  go  through  as 
any  new  communications  technology  develops: 

1.  We  learn  to  accept  the  medium  (over  time) . 

2.  We  become  comfortable  with  the  technology. 

3.  We  begin  to  use  the  medium  creatively. 

4.  The  new  medium  becomes  a  major  form  of 
communications,  we  start  to  rely  on  it  (1:223) . 

In  the  context  of  this  development,  the  Air  Force  is 
somewhere  between  phase  one  and  two.  As  the  use  of 
electronic  transfer  of  data  in  the  commercial  sector 
continues  to  rise,  a  similar  increase  in  the  electronic 
delivery  of  data  is  expected  within  the  DOD. 

Subobiective  2.  Identify  the  current  requirements 
concerning  digital  delivery  of  contractor  data  and  the 
anticipated  benefits  of  using  digital  data  versus  paper 
copies. 

The  current  requirements  to  use  digital  de''  ivery  of 
data  are  specified  in  DODI  5000.2,  "Defense  Acquisition 
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Management  Policies  and  Procedures."  The  direction  is 
further  detailed  in  the  "Computer  Aided  Acquisition 
Logistics  Support  (CALS)  handbook"  (MIL-HDBK-59 ) ,  which 
serves  as  the  standard  to  follow  when  applying  digital 
delivery  of  data  to  a  contract.  Service  and  command  level 
regulations  concerning  management  of  contractor  data  are 
still  being  revised  to  reflect  changes  in  DODI  5000.2.  Most 
of  the  guidance  in  the  area  of  electronic  delivery  of  data 
is  in  the  transfer  of  technical  data. 

The  benefits  of  electronic  delivery  of  data  discussed 
as  part  of  the  mail  survey  and  telephone  interview  are 
consistent  with  those  described  in  the  literature  review. 
Particularly,  speed  of  transmission,  flexibility, 
manipulation  of  data  in  the  received  documents,  and  ease  of 
document  retransmittal  were  all  mentioned  by  respondents  as 
benefits  of  using  electronic  delivery  of  data. 

Subobiective  3.  Identify  problem  areas  or  perceived  problem 
areas  which  prevent  the  effective  use  of  electronic  data 
delivery. 

This  study  revealed  that  there  are  three  main  problems 
areas,  two  are  legitimate  concerns  and  the  other  is  a 
perceived  problem.  The  three  problems  are: 

1.  there  is  a  lack  of  resources  or  capabilities  for 
the  government  to  accept  electronic  delivery  of  data. 

2.  there  is  a  lack  of  training  in  the  area  of 
electronic  communications. 

3.  there  is  a  lack  of  guidance  concerning  the  policy 
and  the  requirements  for  use  of  digital  data  delivery. 


51 


The  lack  of  resources  or  capabilities  appears  to  be  a 
perceived  problem  because  over  95%  of  the  respondents  either 
have  a  computer  on  their  desk  or  share  a  computer  with  other 
in  their  immediate  duty  area.  This  perception  may  exist 
because  throughout  the  interview  process  there  was  the 
expectation  by  respondents  that  electronic  delivery  required 
a  highly  complex,  expensive  computer  system.  With  training 
and  guidance,  this  perception  should  be  overcome. 

Lack  of  training  and  lack  of  guidance  in  the  area  of 
electronic  communications  are  legitimate  concerns.  Almost 
70%  of  the  respondents  received  less  than  two  hours  of 
training  in  the  use  of  E-mail  which  is  the  most  basic  form 
of  electronic  communications.  Additionally,  results 
indicate  that  one  could  expect  with  95%  confidence  that  no 
more  than  42%  of  acquisition  personnel  are  aware  of  the 
current  requirements  regarding  the  digital  delivery  of 
contractor  data. 

Subobiective  4.  Determine  what  lessons  can  be  gained  from 
those  program  offices  that  have  implemented  or  are 
attempting  to  implement  the  electronic  delivery  of 
contractor  data? 

Electronic  delivery  is  still  a  new  approach  for 
receiving  contractor  data.  Therefore,  the  experiences  and 
lessons  learned  are  very  limited.  The  lessons  as  discussed 
by  survey  respondents  primarily  focused  on  the  fact  that  the 
mediums  used  must  be  compatible  between  the  government  and 
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the  contractor.  This  requires  that  the  contract  specify  the 
particular  government  software  and  hardware  with  which  the 
contractor  must  interface  when  data  is  delivered  digitally. 

Considering  the  amount  of  non-CDRL  data  delivered 
through  digital  means  there  appears  to  be  a  desire  on  the 
part  of  both  the  contractor  and  the  government  to  pass  data 
electronically.  A  major  step  toward  realizing  this  mutual 
goal  is  to  formally  require  digital  data  delivery  in  the 
contract. 

Limitations  of  Research 

As  previously  noted,  a  limitation  of  this  research  was 
the  selection  criteria  for  determining  interview  subjects. 
The  selection  criteria  did  not  take  into  account  contracts 
awarded  prior  to  1  October  1989.  Acquisition  personnel  in 
many  of  the  older  major  weapon  system  programs  were  not 
interviewed  because  those  contracts  were  awarded  in  the 
early  to  mid- '80s. 

Another  limitation  of  this  research  was  that  this  study 
focused  only  on  government  acquisition  personnel  at  ASD  and 
HSD.  With  minor  modifications  to  the  interview  guide  and 
survey  questionnaire,  this  study  could  be  performed  on  other 
populations . 

Recommendations 

Future  Studies.  There  are  many  possible  follow-on 
studies  in  the  area  of  electronic  delivery  of  contractor 
data.  Three  recommended  projects  are  described  below. 
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One  area  for  further  research  is  to  perform  a  similar 
study  of  government  contractors  to  determine  the  extent  that 
these  individuals  are  trained  in  data  management,  primarily 
as  it  relates  to  electronic  communications.  It  would  be 
valued>le  to  know  the  extent  to  which  prime  contractors  and 
subcontractors  are  using  digital  delivery  of  data. 

Another  future  study  would  be  to  modify  the  selection 
criteria  for  the  interview  process  to  take  into  account 
modifications  to,  and  options  exercised  on,  major  weapon 
system  contracts  in  the  past  two  years  to  the  present. 

Because  much  of  the  guidance  concerning  electronic 
delivery  of  data  concentrates  on  the  delivery  of  technical 
data,  a  research  study  could  evaluate  the  delivery  process 
for  non-technical  CDRL  items  to  determine  the  extent  that 
this  data  could  be  delivered  electronically  to  the 
government  using  existing  communications  methods. 

General  Recommendations.  The  fax  machine  has  become  a 
common  link  to  transmit  data  quickly  between  the  government 
and  the  contractor  and  allows  information  to  be  passed  more 
quickly.  However,  the  use  of  the  fax  has  increased  the 
amount  of  paper  generated  while  it  is  still  necessary  to 
send  hardcopies.  Because  a  majority  of  the  individuals 
already  use  E-mail,  it  is  recommended  that  E-mail  be  used  to 
transmit  much  of  the  correspondence  (memos  and  advance 
copies  of  contract  letters)  that  is  currently  being 
delivered  via  a  fax.  If  a  contractor  does  not  have  E-mail 
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capability,  the  government  should  investigate  the 
installation  of  fax^servers  on  its  current  LAN  systems. 

It  is  also  recommended  that  the  requirements  and 
guidance  for  the  digital  delivery  of  data  specifically 
address  non-technical  data  items  as  well  as  technical  data. 
This  is  recommended  because  non-technical  data  items  could 
be  more  easily  transmitted  using  existing  e-mail 
capabilities. 

Additionally,  many  individuals  already  have  access  to  a 
LAN.  However,  a  majority  of  these  individual  have  received 
little  or  no  training  in  the  use  of  the  LAN.  Therefore,  it 
is  recommended  that  as  individuals  are  provided  access  to  a 
LAN,  they  are  also  provided  the  necessary  training. 

It  is  also  recommended  that  current  requirements  and 
guidance  concerning  the  digital  delivery  of  contractor  data 
should  be  provided  to  acquisition  personnel,  including 
support  contractor  personnel. 
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LSY  (Capt  Farkas,  DSN:  785-8989)  3  Jul  91 

Electronic  Deliveiry  of  Contractor  Data  Suirvey  Package 


Survey  Recipient: 


1.  Please  take  the  time  to  complete  the  attached 
questionnaire  and  return  it  in  the  enclosed  envelope  by  17 
July  1991. 

2.  This  survey  examines  the  extent  to  which  contractor  data 
is  being  submitted  by  electronic  means  on  government 
contracts.  The  data  we  gather  will  become  part  of  an  AFIT 
research  project  and  may  influence  the  way  contractors 
submit  data  in  the  future.  Your  individual  responses  will 
be  combined  with  others  and  will  not  be  attributed  to  you 
personally. 

3.  Your  participation  is  completely  voluntary,  but  we  would 
certainly  appreciate  a  few  minutes  of  your  valuable  time. 

For  further  information  contact  Capt  Grant  at  DSN:  785-3355. 


Arthur  A.  Hunguia  2  Atch 

Assoc  Prof  of  Systems  Management  1. 

Questionnaire 

School  of  Systems  and  Logistics  2.  Return 

Envelope 
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D8AF-8CN-91-49 
Expires  30  8ep  91 

Electronic  Delivery  of  Contractor  Data  8urvey 

The  following  survey  contains  a  combination  of  multiple 
choice  and  short  answer  questions.  Please  read  each 
question  carefully  and  then  record  your  response  directly  on 
the  survey  form.  If  you  need  more  space  to  answer  a 
particular  question,  feel  free  to  use  the  reverse  side  of 
the  survey  form.  Thank  you  for  your  time. 

1.  Please  list  your  specific  job  title  and  position. 


2.  What  is  your  current  rank  or  grade  level?  (Please 
circle  one) 


0-1 

to 

0-3 

0-4 

to 

0-5 

E-1 

to 

E-5 

E-6 

to 

E-9 

GS4 

to 

GS5 

GS6 

to 

GS7 

GS8 

to 

GSll 

GS12 

to 

GS13 

Other:  _ 

(please  clarify) 


3.  How  long  have  you  been  working  in  data  management? 
(Please  circle  one) 


a.  less  than  1  year 

b.  1  to  2  years 

c.  2  to  3  years 

d.  3  to  4  years 

e.  more  than  4  years 

4.  How  much  of  your  work  time  is  spent  working  in  data 
management?  (Please  circle  one) 


a.  less  than  5% 

b.  5-10  percent 

c.  10-25  percent 

d.  25  to  50  percent 

e.  50  to  75  percent 

f.  greater  than  75  percent 
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5.  What  data  management  related  courses  have  you  attended?  When? 
(Please  drde  the  letter  corresponding  to  each  course  you  have  taken  and 
provide  the  month  and  year  you  took  the  course  in  the  space  provided) 


a.  SYS  370  Defense  Data  Management 

b.  SYS  150  Engineering  Data  Management 

c.  Other; 

6.  Consider  only  the  time  you  spend  working  on  data  management 
activities.  Please  allocate  the  time  you  spend  to  each  of  the  following 
data  management  activities.  (Please  estimate  the  percent  of  time  spent  on 
each  activity  and  record  the  percent  in  the  space  provided  next  to  each  activity. 
The  total  time  allocated  should  sum  to  100%) 

Data  Management  Activitv  %  Time 

a.  Data  requirements  definition  (AF  Form  585  preparation) 

b.  Data  entry  (AF  Form  1423  preparation,  input  tracking  data 

c.  Data  tracking 

d.  Data  review 

e.  Other: 


7.  Which  of  the  following  requirements  are  specified  by  OODi  5000.2 
concerning  the  electronic  delivery  of  contract  data  deliverables?  (Please 
circle  the  correct  response) 

a.  Contract  deliverable  data  will  be  prepared  and  used  in  digital  form  on 
contracts  for  computer  resources  with  total  costs  of  greater  than  $2 
million. 

b.  Contract  deliverable  data  will  be  prepared  and  used  in  digital  form  on  all 
contracts  following  the  Milestone  1  decision. 

c.  Contract  deliverable  data  in  digital  form  will  be  prepared  and  used  with 
the  approval  of  the  base  communications  officer  on  all  contracts  awarded 
after  30  September  1990. 

d.  Contract  deliverable  data  will  be  prepared  and  used  in  digital  form  unless 
it  is  not  cost-effective  for  the  government. 

e.  All  of  the  above. 

f.  I  don’t  know. 
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8.  Describe  the  computer  resources  available  to  you  at  work.  (Please 
circle  all  appropriate  responses) 

a.  Personal  computer  on  my  desk. 

b.  I  share  a  personal  computer  with  others  in  my  duty  area. 

c.  I  have  a  Vax  (mainframe)  terminal  in  my  immediate  duty  area. 

d.  There  are  no  computer  resources  available  for  my  use. 

e.  Other  (explain): 


9.  How  many  hours  of  formal  training  have  you  had  in  the  use  of  E-mail? 
(Please  circle  one) 

a.  none.  b.  1-2  hours 

c.  3-4  hours  d.  5-6  hours 

e.  7-8  hours  f.  greater  than  8  hours. 


10.  How  often  do  you  use  Electronic-Mail  (E-Mail)?  (Please  circle  one) 

a.  At  least  once  a  day. 

b.  At  least  once  a  week. 

c.  At  least  twice  a  month. 

d.  Once  a  month  or  less  frequently. 


11.  What  is  your  E-Mail  user  name?  (Please  write  your  user  name  on  the 
space  provided,  (f  you  are  unsure,  please  explain) 


12.  Do  you  use  a  computer  for  data  management?  (Please  circle  one) 
a.  Yes  b.  No 
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13.  If  you  do  not  use  a  computer  for  data  management,  what  are  the 
primary  reasons  you  do  not?  (Please  list  the  reasons  you  do  not  use  a 
computer  for  data  management  in  the  space  provided) 


14.  Do  you  have  access  to  a  Local  Area  Network  (LAN)  within  your  duty 
area?  (Please  circle  one) 

a.  Yes 

b.  No 

c.  1  don't  know  if  I  have  access  to  a  LAN. 

d.  I  don’t  know  what  a  LAN  is. 


15.  How  much  training  have  you  received  in  the  use  of  Local  Area 
Networks  (LANs)?  (Please  circle  one) 

a.  less  than  4  hours 

b.  4-8  hours 

c.  8-12  hours 

d.  12-16  hours 

e.  greater  than  16  hours 

f.  none. 


16.  Are  you  working  on  any  programs  that  require  electronic  delivery  of 
data?  (Please  circle  one) 


a.  Yes 


b.  No  (go  to  question  20) 


17.  Why  was  the  electronic  delivery  of  data  used?  (Please  explain) 


18.  What  benefits  are  you  experiencing  due  to  the  electronic  delivery  of 
contractor  data?  (Please  describe) 


19.  What  are  the  drawbacks  you  have  experienced  due  to  the  electronic 
delivery  of  contractor  data?  (Please  describe) 
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20.  Why  is  electronic  delivery  of  contractor  data  not  being  used?  (Please 
explain) 


21.  Are  you  receiving  from  the  contractor  or  transmitting  to  the  contractor 

other  contract  related  correspondence  using  electronic  means? _ 

If  yes,  what  type  of  data  and  how  is  it  transmitted/received? 


1.  Are  you  using  electronic  delivery  of  data  items  on  contract  #. 

_ ?  YES  or  NO? 

If  no  go  to  Question  10. 


2.  Is  the  government  receiving  all  contract  deliverables  through  electronic 
means?  YES  or  NO? 

If  yes,  go  to  Question  5. 


3.  Which  data  items  are  you  receiving  electronically? 


4.  Why  aren’t  all  data  deliverables  received  electronically? 


5.  How  *ire  you  receiving  the  data  deliverables?  If  difficult  to  answer,  the 
following  examples  are  provided.  Also,  briefly  describe  the  process. 

a.  Through  the  Defense  Data  Network  (DDN) 

b.  Through  a  dedicated  terminal. 

c.  Through  a  Facsimile  (FAX)  machine. 

d.  On  magnetic  tape  or  floppy  disk. 

e.  Other. 


9.  Are  you  receiving  from  the  contractor  or  transmitting  to  the  contractor  other 
contract  related  correspondence  using  electronic  means?  YES  or  NO?  If 
yes,  what  type  of  data  and  how  is  it  transmitted/received? 


10.  Why  was  the  electronic  delivery  of  contractor  data  not  used  on  this 
contract? 


1 1 .  Are  you  receiving  from  the  contractor  or  transmitting  to  the  contractor  other 
contract  related  correspondence  using  electronic  means?  YES  or  NO?  If 
yes,  what  type  of  data  and  how  is  it  transmitted/received? 


1.  Pl«ase  list  your  specific  job  title  and  position. 

Data  Management  39  Logistics  1 
Configuration  Management  30  Engineering  3 
Program  Management  3  Other  2 


2.  What  is  your  current  rank  or  grade  level?  (Please 
circle  one) 

Responses 


a.  0-1 

to 

0-3 

5 

b.  0-4 

to 

0-5 

1 

c.  E-1 

to 

E-5 

0 

d.  E-6 

to 

E-9 

1 

e.  GS4 

to 

GS5 

6 

f.  GS6 

to 

GS7 

5 

g.  GS8 

to 

GSll 

11 

h.  GS12 

to 

GS13 

23 

i .  other 

26 

Total  number  of  respondents 

78 

3.  How  long  have  you  been  working  in  data  management? 

(Please  circle  one) 

Responses 


a.  less  than  1  year  10 

b.  1  to  2  years  9 

c.  2  to  3  years  7 

d.  3  to  4  years  10 

e.  more  than  4  years  37 

f.  no  response  5 


4.  How  much  of  your  work  time  is  spent  working  in  data 
management?  (Please  circle  one) 

Number  of  Responses 


a.  less  than  5%  13 

b.  5-10  percent  7 

c.  10  -  25  percent  14 

d.  25  to  50  percent  10 

e.  50  to  75  percent  5 


f.  greater  than  75  percent  30 
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5.  What  data  managament  related  courses  have  you  attended? 
When?  (Please  circle  the  letter  corresponding  to  each 
course  you  have  taken  and  provide  the  month  and  year  you 
took  the  course  in  the  space  provided.) 

Responses 

a.  SYS  370  Defense  Data  Management  46  yes  32  no 

b.  SYS  225  Engineering  Data  Management  16  yes  62  no 

c.  Other:  20  yes  58  no 

6.  Consider  only  the  time  you  spend  working  on  data 
management  activities.  Please  allocate  the  time  you  spend 
to  each  of  the  following  data  management  activities. 

(Please  estimate  the  percent  of  time  spent  on  each  activity 
and  record  the  percent  in  the  space  provided  next  to  each 
activity.  The  total  time  allocated  should  sum  to  100%) 


6a.  0% 

-  22 

6b.  0% 

-  36 

6C.  0% 

- 

19 

1% 

-  1 

2% 

- 

2 

1% 

- 

1 

2% 

-  2 

3% 

- 

2 

3% 

- 

1 

3% 

-  2 

5% 

- 

2 

5% 

- 

1 

5% 

-  14 

10% 

- 

6 

10% 

- 

9 

10% 

-  11 

15% 

- 

2 

15% 

- 

4 

15% 

-  7 

20% 

- 

6 

20% 

- 

4 

20% 

-  3 

25% 

- 

6 

25% 

mm 

10 

25% 

-  2 

30% 

- 

1 

30% 

- 

7 

30% 

-  4 

40% 

- 

2 

35% 

- 

1 

40% 

-  1 

45% 

- 

2 

40% 

- 

1 

50% 

-  1 

48% 

- 

1 

45% 

- 

4 

100% 

-  1 

50% 

- 

3 

50% 

- 

6 

no  ans. 

-  7 

60% 

- 

1 

55% 

- 

1 

no  ans. 

- 

7 

70% 

- 

1 

no  ans. 

- 

7 

6d.  0%  - 

19 

6e.  0% 

-  9 

5%  - 

8 

5% 

-  3 

10%  - 

13 

10% 

-  4 

6e.  90%  - 

2 

15%  - 

2 

20% 

-  2 

95%  - 

1 

20%  - 

6 

25% 

-  1 

100%  - 

3 

25%  - 

10 

25% 

-  1 

no  ans  - 

6 

30%  - 

2 

30% 

-  1 

40%  - 

7 

40% 

-  3 

45%  - 

4 

45% 

-  1 

50%  - 

4 

50% 

-  0 

80%  - 

2 

60% 

-  2 

90%  - 

2 

70% 

-  1 

97%  - 

2 

75% 

-  3 

no  ans.  -  7 

80% 

-  2 
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7.  Which  of  the  following  requirements  are  specified  by  DODI  5000.2 
concerning  the  electronic  delivery  of  contract  data  deliverables?  (Please 
circle  the  correct  response.) 

a.  Contract  deliverable  data  will  be  prepared  and  used  in  digital  form  on  all 
contracts  with  total  costs  of  greater  than  $2  million. 

b.  Contract  deliverable  data  will  be  prepared  and  used  in  digital  form  on  all 
contracts  following  the  Milestone  1  decision. 

c.  Contract  deliverable  data  in  digital  form  will  be  prepared  and  used  on  all 
contracts  awarded  after  30  September  1990. 

d.  Contract  deliverable  data  will  be  prepared  and  used  in  digital  form  unless 
it  is  not  cost-effective  for  the  government. 

e.  All  of  the  above. 

f.  I  don’t  know. 

Responses:  a.  1  b.  0  c.  2  d.  23  e.  5  f.  46 
no  ans.  1 


8.  Describe  the  computer  resources  available  to  you  at  work.  (Please 
circle  all  appropriate  responses) 

a.  Personal  computer  on  my  desk. 

b.  I  share  a  personal  computer  with  others  in  my  duty  area. 

c.  I  have  a  Vax  (mainframe)  terminal  in  my  immediate  duty  area. 

d.  There  are  no  computer  resources  available  for  my  use. 

e.  Other  (explain): 


Responses:  a.  54  yes  24  no 

b.  20  yes  58  no 

c.  16  yes  62  no 

d.  2  yes  76  no 

e.  4  yes  74  no 
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9.  Hov  many  hours  of  formal  training  hava  you  had  in  the 
uss  of  B-nail?  (  Please  circle  one) 


Responses 


a.  none 

32 

b.  1-2  hours 

24 

c.  3-4  hours 

13 

d.  5-6  hours 

0 

e.  7-8  hours 

5 

f.  greater  than  8 

hours  4 

10.  Hov  often  do  you  use  Electronic-Mail  (E-Mail)?  (Please 
circle  one) 

Responses 


a.  At  least  once  a  day.  46 

b.  At  least  once  a  week.  8 

c.  At  least  twice  a  month.  5 

d.  Once  a.  month  or  less  frequently. 16 

no  answer  3 


11.  What  is  your  E-Mail  user  name?  (Please  write  your  user 
name  on  the  space  provided.  If  you  are  unsure,  please 
explain) 

Responses:  68  knew  their  user  name 

6  did  not  know  their  user  name 
4  did  not  answer  this  question 


12.  Do  you  use  a  computer  for  data  management?  (Please 
circle  one) 

Responses:  59  yes 
18  no 

1  no  answer 


13.  If  you  do  not  use  a  computer  for  data  management,  what 
are  the  primary  reasons  you  do  not?  (Please  list  the 
reasons  you  do  not  use  a  computer  for  data  management  in  the 
space  provided) 
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14.  Do  you  have  access  to  Local  Area  Network  (LAN)  within  your  duty 
area?  (Please  circle  one) 

a.  Yes 

b.  No 

c.  I  don’t  know  if  I  have  access  to  a  LAN. 

d.  I  don’t  know  what  a  LAN  is. 

Responses 

a.  56 

b.  4 

c.  13 

d.  2 

3  no  answer 


15.  How  much  training  have  you  received  in  the  use  of  Local  Area 
Networks  (LANs)?  (Please  circle  one) 


Responses 

a. 

less  than  4  hours 

21 

b. 

4-8  hours 

7 

c. 

8-12  hours 

1 

d. 

12-16  hour 

3 

e. 

greater  than  16  hours 

4 

f. 

none. 

40 

2  no  answer 


16.  Are  you  working  on  any  programs  that  require  electronic  delivery  of 
data?  (Please  circle  one) 


Responses 

a.  Yes  11 

b.  No  (go  to  question  20)  67 
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17.  Why  was  the  electronic  delivery  of  data  used?  (Please 

explain) 

Responses: 

1.  To  reduce  paperwork. 

2.  It’s  required. 

3.  No  answer.  (II) 

4.  For  speed,  reduction  in  cost,  and  ease  of  filing. 

5.  Information  concerning  service  reports  can  go  directly  Into  data  base 
Cinfo  center). 

6.  Management  of  data  is  enhanced  and  is  made  available  to  all. 

7.  Able  to  manipulate  the  data  when  changes  are  needed. 

8.  We  are  procuring  a  large  data  intensive  automated  training  system 
which  will  have  all  of  the  documents  stored  electronically  on  the  system.  Most 
of  the  CDRLS  will  be  used  for  system  maintenance  and  must  be  electronically 
stored.  We  also  have  a  direct  electronic  line  established  with  the  contractor. 

18.  What  benefits  are  you  experiencing  due  to  the  electronic  delivery  of 
contractor  data?  (Please  describe) 

Responses: 

1.  Not  sure. 

2.  None  yet.  not  on  contract. 

3.  No  answer.  (1) 

4.  Speed,  reductions  in  cost  and  ease  of  filing. 

5.  Receiving  drawings  on  magnetic  tape  and  90%  of  service  reporting  is 
accomplished  through  computer  generated  E-MAIL. 

6.  Anyone  requiring  a  copy  can  get  one. 

7.  More  flexibility. 

8.  Direct  link  is  quicker  than  mail.  Feedback  on  draft  correspondence 
can  be  done  on-line  and  quickly.  Documents  can  be  imported  to  other 
documents.  Part  or  as  a  whole,  so  that  wording  can  be  revised  and  consistent. 
Can  transmit  documents  as  is  via  DDN  to  other  organizations.  Not  as  much 
paper  to  shuffle.  Easy  to  reproduce.  Helps  toward  achieving  CALS 
compliance. 
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19.  What  ara  the  drawbacks  you  have  experienced  due  to  the 
electronic  delivery  of  contractor  data?  (Please  describe) 

Responses : 

1.  It's  too  new.  Need  procedures  and  guidance  in  the 
following  areas: 

a.  Dues  access  constitute  delivery? 

b.  Use  of  contractor  system  with  government  access 
to  read  versus  transfer  to  government  computer. 

c.  Compatibility  of  government  terminals  with 
contractor's  computer  media. 

d.  Need  for  1423  versus  contractor  system  under 
which  information  is  "available". 

2.  Difficult  to  review  if  no  hardcopy  is  availedsle. 

3.  Specific  software  packages  and  version  numbers  need 
to  be  specified  in  the  DID. 

4.  AF  contracts  offices  don't  like  it  very  much. 
They  like  transmittals  going  from  company  to  company  on 
paper  with  real  signatures.  Must  be  very  specific  about  the 
electronic  format  selected.  Choose  what's  best  for  you, 
i.e.  ASCII,  Word  Perfect,  Enable,  etc.  Have  to  be  careful 
with  document  configuration  management,  documents  are  easy 
to  change. 
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20.  Why  is  slectronle  delivery  of  contractor  data  not  being 
used?  (Please  explain) 


Table  4. 

Question  20  Svunmary  of  Results 


REASON 

NUMBER  OF  RESPONSES 

No  Capability 

1 

Found  not  cost  effective 

4 

No  Answer 

11 

Old  Programs 

15 

Not  currently  using,  will 
be  in  the  future 

9 

Not  completely  hooked  up 

1 

Don't  know 

13 

Not  considered 

3 

Need  guidance 

4 

No  uniform  medium 

1 

Resources  not  available 

3 

Problem  making  hardcopies 

1 

Prefer  hardcopy 

1 

21.  Are  you  receiving  from  the  contractor  or  transmitting 
to  the  contractor  other  contract  related  correspondence 
using  electronic  means?  If  yes,  what  type  of  data  and  how 
is  it  transmitted/received? 


E-MAIL 


nr.  of  responses 


correspondence  4 
schedule  2 
AFTO  Form  349s  1 
some  CDRL  items  1 
Service  reports  3 


Draft  correspondence  of  alx  kinds  as  well  as  informal 
notes  and  messages  (direct  links  to  the  contractor) . 

Graphics  files  and  other  executable  programs  are  received  as 
well. 
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FAXING 


Nr .  of  Responses 


1423s  (1) 

schedule  (2) 

some  CDRLs  (1) 

when  timely  (2) 

LTRs/memos  (2) 

Draft  -  minutes/ agendas  (1) 
letters 

DISK/TAPE 

miscellaneous  (1) 

some  CDRL  items  (1) 
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Question  1. 

Response  :  7  YES  102  NO  17  Unable  to  contact 

2.  Is  the  government  receiving  all  contract  deliverables  through  electronic 
means?  If  yes,  go  to  Question  5. 

Response:  2  YES  5  NO 

3.  Which  data  items  are  you  receiving  electronically? 

Response: 

software  configuration  Kerns 
software  documentation 
provisioning 
test  specifications 

software  code  brought  in  eiectronicaliy 
ali  data  (2  respondents) 

only  post  production  support  analysis,  which  is  a  list  of  tests  for  in- 
house  production  and  testing 
scheduies  and  drawings 

4.  Why  aren't  all  data  deliverables  received  electronically? 

Response:  No  standardization 

5.  How  are  you  receiving  the  data  deliverables?  If  difficult  to  answer,  the 
following  examples  are  provided.  Also,  briefly  describe  the  process. 

a.  Through  the  Defense  Data  Network  (DDN) 

b.  Through  a  dedicated  terminal. 

c.  Through  a  Facsimile  (FAX)  machine. 

d.  On  magnetic  tape  or  floppy  disk. 

e.  other. 

Responses: 

5  magnetic  tape  or  floppy  disk  file  transfer 
1  file  transfer  using  the  VAX  (still  working  on  becoming  compatible 
with  the  contractor’s  system) 

1  through  a  dedicated  terminal 
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6.  What  benefits  have  you  realized  using  electronic  delivery  of  data  items? 

Response: 

•  less  paperwork 

•  saving  time 

.  speed  of  deiivery,  even  faster  than  express  maii 

-  turn  around  time  to  review  CDRL’s  and  to  get  comments  back  to 
contractor  improved,  this  includes  being  able  to  more  easily  deal 
with  the  ALC’s  to  get  comments. 

-  flexibility  to  take  contractor  data  and  add  it  to  larger  reports 

•  more  expedient 

7.  What  drawbacks  have  you  experienced  using  electronic  delivery  of  data 
items? 

Response: 

Not  having  an  integrated  package  that  all  parts  of  the  Air  Force  are 
familiar  with. 

Contractor  not  compatible  with  ALC. 

Still  too  new  •  unable  to  comment. 

As  long  as  systems  are  compatible  there  haven’t  been  any  problems. 
The  contractor  became  compatible  with  the  government. 

Technical  glitches  of  down  time;  however,  still  faster  than  using  mail. 

8.  What  would  you  do  differently  concerning  the  delivery  of  contractor  dala? 
Response: 

If  the  government  imposes  requirements  then  they  should  provide  a 
complete  integrated  package  to  the  contractor. 

3  Nothing  yet. 

2  Still  too  new! 

9.  Are  you  receiving  from  the  contractor  or  transmitting  to  the  contractor 
other  contract  related  correspondence  using  electronic  means?  If  yes,  wrfiat 
type  of  data  and  how  is  it  transmitted/received? 

The  response  to  this  question  has  been  included  with  question  11  findings 
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10.  Why  was  the  electronic  delivery  of  contractor  data 
not  used  on  this  contract? 

number  of  Responses 

1.  Many  CDRLs  are  classified.  1 

2.  Too  expensive.  1 

3.  Still  need  hardcopy.  2 

4.  Compatibility.  1 

5.  Not  set  up.  2 

6.  No  reason.  11 

7.  Small  businesses.  17 

8.  No  CDRLs.  5 

9.  Not  considered.  3 

10.  Few  CDRLs.  4 

11.  Not  required.  9 

12.  Not  trained.  14 

13.  FM8  contractor.  1 

14.  Rushed  to  get  on  contract.  2 

15.  Commercial  nature  of  program.  1 

16.  Mot  aware  of  capabilities.  7 

17.  Didn't  know.  14 

18.  Small  Business  Research  Project.  9 

11.  Are  you  receiving  from  the  contractor  or 
transmitting  to  the  contractor  other  contract  related 
correspondence  using  electronic  means?  If  yes,  what  type  of 
data  and  how  is  it  transmitted/received? 

Responses : 

Fax  Disk  E-Mail 

104  yes  5  no  9  yes  2  yes 


Fax; 

1. 

2. 

3. 

4. 

5. 

6. 


For  advance  copies.  100 
Anything  but  formal  submittals.  1 
General  correspondence  (agendas,  minutes) . 
Only  when  necessary  to  expedite.  1 
MODS.  2 

Both  draft  and  formal.  4 


Disk: 

1.  Pricing  information.  2 

2.  Proposals  (more  efficient,  manipulation  of  data).  2 

3.  Draft  specifications,  corrections  made  and  returned 
for  formal  submittal.  1 


E-Mail: 

1.  Not  getting  alot  of  usage. 

Dedicated  terminal: 

1.  Everything  Is  sent  over  this  terminal.  The  government  has 
limited  access  to  view  the  contractor's  data. 
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NS*.  7540-0- -280-5500 


AFIT  Control  Number  AFIT/GSM/LSY/91S-9 


AFIT  RESEARCH  ASSESSMENT 


The  purpose  of  this  questionnaire  Is  to  determine  the  potential  for  cur¬ 
rent  and  future  applications  of  AFIT  thesis  research.  Please  return 
completed  questionnaires  to:  AFIT/LSC*  Hright-Patterson  AFB  OH 
45433-6583. 

1.  Old  this  research  contribute  to  a  current  research  project? 
a.  Yes  b.  No 


2.  Do  you  believe  this  research  topic  Is  significant  enough  that  It  would 
have  been  researched  (or  contracted)  by  your  organization  or  another 
agency  If  AFIT  had  not  researched  It? 

a.  Yes  b.  No 


3.  The  benefits  of  AFIT  research  can  often  be  expressed  by  the  equivalent 
value  that  your  agency  received  by  virtue  of  AFIT  performing  the  research. 
Please  estimate  what  this  research  would  have  cost  In  terms  of  manpower 
and/or  dollars  If  It  had  been  accomplished  under  contract  or  If  It  had 
been  done  In-house. 

Man  Years _  $ _ 


4.  Often  It  Is  not  possible  to  attach  equivalent  dollar  values  to 
research,  although  the  results  of  the  research  may.  In  fact,  be  Important. 
Whether  or  not  you  were  able  to  establish  an  equivalent  value  for  this 
research  (3  above),  what  Is  your  estimate  of  Its  significance? 

a.  Highly  b.  Significant  c.  Slightly  d.  Of  No 

Significant  Significant  Significance 


5.  Comments 


Marne  and  Grade  Organization 

Position  or  Title  Address 


